S7 Behaviour sensitivity: model parameters

Introduction

This document presents the results assessing the impact of input assumptions on model behaviour for the Decision Support tool for Child and Adolescence Obesity, a system dynamic model developed to explore underlying relationships that contribute to youth obesity.

Outline of behaviour sensitivity analysis

System dynamics models are described as being ‘causally-descriptive’, meaning that unlike other ‘black-box’ models, it is not sufficient that system dynamics only produce accurate output. In addition to being numerically valid, the behavior of system dynamics models needs to be validated.

The test presented in this document examines which input assumptions have the greatest impact on model behavior, unlike previous sensitivity analyses, which focus on numerical relationships between input assumptions and outputs (either model outcomes or policy outcomes).

Methods

A sensitivity analysis was conducted, testing input variables between \(\pm\) 20% of their initial values. While each input value may have more extreme values, these inputs are considered to be outside probable ranges. For each sensitivity analysis, the model generates 78 BMI prevalence outcomes, including 3 BMI categories, 13 age groups, and two genders.

A full depiction of the parameters tested can be found in the S5 Extreme conditions test: model parameters and S6 Extreme conditions test : policy parameters online supplementary materials.

Model behaviour refers to the trend, phase, lag, or amplitude of a system dynamics model result over modelled time. Inputs that impact these outcome characteristics are considered to influence the model’s behaviour.

Outcome Measure

The behavior of each model test was evaluated using the cumulative difference of the first derivative. The following plots explain how this outcome measure is calculated.

Figure panel A shows a plot of three outcomes: the base model, with no changes to any initial assumptions; the result varies an input assumption that produces similar model behavior; and a second test, where varying the input assumption produces different model behavior. The absolute cumulative difference between the model outcomes is plotted in panel C.

Panel B plots the first derivative of each model result. The model results with similar behavior also have a similar first derivative. Conversely, the different behavior result has differences in the derivative. The cumulative absolute difference of the first derivative shows a greater difference between the two model results, highlighting the difference between behaviors.

While the cumulative absolute difference measure cannot give insights into the model structure, it is useful to screen input assumptions that impact model behavior.

Unlike the extreme test, this analysis focuses on determining which input assumption impacts the model behavior.

Example plot for model behaviour

Example plot for model behaviour

How to interpret the results

In this analysis, each input assumption is systematically sampled 20 times within the range of +/- 20% of the initially assumed input value. The cumulative difference of the first derivative is utilized to evaluate how each variation in the input variable influences model behavior. The average cumulative difference across the 20 samples is then summarized in the result plots.

The following figure outlines the process of generating the summary plots. Subplot A displays the average cumulative difference for the 20 samples of the metabolic equivalence of task (MET) for vigorous physical activity, presented for all tested variables.

The primary outcome, the prevalence of obesity, is depicted in subplot B for each assumption of vigorous MET value across different age groups. The subsequent derivative is illustrated in subplot C, and the cumulative difference in the derivative is shown in subplot D, with the average incremental difference highlighted in red. This average cumulative difference in the first derivative serves as the summary value presented in the summary plots.

A higher average cumulative difference in the first derivative indicates a variable with a more substantial impact on model behavior.

Results

Overall Ranking of sensitivity

The 649 input assumptions tested have been aggregated and summarized into 70 variables by calculating the mean of the cumulative difference in derivatives for age-gender-BMI specific inputs. These 70 overarching variables are ranked in the table below.

Here is the ranking of aggregate input assumptions, where input assumptions with multiple subgroups have been averaged. For instance, changes in body weight are tested by age-gender-BMI subgroups, but the ranking is based on the mean cumulative difference in first derivatives.

The findings indicate that variables with a global impact, influencing the entire modeled population, have the most significant impact on model behavior.

Label Average cumulative difference in derivatives
SUGAR Kj per gram 0.8977340
FATS Kj per gram 0.8944223
“NON-SUGAR CARBOHYDRATES Kj per gram” 0.8195918
INACTIVE METs 0.7031066
SLEEP METs 0.6844801
PROTEIN Kj per gram 0.5855636
SCREEN TIME METs 0.4285511
VIGOROUS PA METs 0.4048512
LIGHT PA METs 0.3383088
MODERATE PA METs 0.2971283
FATS TEF 0.1917250
CARBOHYDRATE TEF 0.1718966
PROTEIN TEF 0.1655541
Grains Reported Intake INTERCEPT 0.1599997
Dairy Reported Intake INTERCEPT 0.1541384
Discretionary foods Reported Intake INTERCEPT 0.1417191
Daily SLEEP minutes Reported INTERCEPT 0.1117347
Proportion of nutrients within food group Inputs 0.1012992
SUGAR TEF 0.0908475
Daily VIGOROUS PA minutes Reported INTERCEPT 0.0903123
Meat and Protein Reported Intake INTERCEPT 0.0832654
Discretionary foods Reported Intake AGE SLOPE 0.0822170
Years to achieve change 0.0802966
Schofield Equation Coefficient 0.0675960
“Body Weight (kg)” 0.0618553
Fruit Reported Intake INTERCEPT 0.0535116
Schofield Equation intercept 0.0478507
Daily MODERATE PA minutes Reported INTERCEPT 0.0462249
Meat and Protein Reported Intake AGE SLOPE 0.0421011
Grains Reported Intake AGE SLOPE 0.0385217
Other Reported Intake INTERCEPT 0.0372783
Vegetables Reported Intake INTERCEPT 0.0309869
Daily LIGHT PA minutes Reported INTERCEPT 0.0272870
Other Beverages Reported Intake INTERCEPT 0.0272579
Sugar based beverage Reported Intake AGE SLOPE 0.0261384
Sugar based beverage Reported Intake INTERCEPT 0.0252786
Fats and Oils Reported Intake INTERCEPT 0.0249171
Other Beverages Reported Intake AGE SLOPE 0.0147834
Other Reported Intake AGE SLOPE 0.0132773
Vegetables Reported Intake AGE SLOPE 0.0123815
Daily VIGOROUS PA minutes Reported AGE SLOPE 0.0113788
Initial BMI Prevalence Inputs 0.0095932
Dairy Reported Intake AGE SLOPE 0.0072566
Fats and Oils Reported Intake AGE SLOPE 0.0063582
Daily MODERATE PA minutes Reported AGE SLOPE 0.0063546
“Reference height (m)” 0.0053184
Fruit Reported Intake AGE SLOPE 0.0036740
Daily LIGHT PA minutes Reported AGE SLOPE 0.0035969
Daily SLEEP minutes Reported AGE SLOPE 0.0032736
“Growth Function kJ/day” 0.0031015
Adult to Child Social Transmission PAL Behaviors 0.0022373
Child to Child Social Transmission of PAL Behaviors 0.0018585
Parents BMI 0.0017146
Initial FM % 0.0009904
“Percentage BF >6mths reported” 0.0009100
Intercept 0.0004478
BMI Hazards Ratios 0.0002142
Adult to Adult Social Transmission of PAL Behaviors 0.0001529
“Percentage non-core > 0 reported” 0.0000676
Non-core >0 0.0000650
“Percentage TV >1 per day reported” 0.0000486
TV >=1 0.0000464
Breastfeeding >=6mth 0.0000272
Adult to Child Social Transmission DIET Behaviors 0.0000003
Child to Child Social Transmission of DIET Behavior 0.0000002
Adult to Adult Social Transmission of DIET Behaviors 0.0000000
Daily SCREEN TIME minutes Reported INTERCEPT 0.0000000
Daily SCREEN TIME minutes Reported AGE SLOPE 0.0000000
Water Reported Intake AGE SLOPE 0.0000000
Water Reported Intake INTERCEPT 0.0000000
Note:
Ranked variables

Examining the most sensitive variable: energy density of sugar

Body Weight

Change in Male Body Weight input assumptions

Change in Female Body Weight input assumptions

Height

Change in Male height input assumptions

Change in Female height input assumptions

Schofield Equation - Intercept

Schofield Equation - Ceofficient

Growth Function kJ/day

Macronutrient energy density

Thermic effect of food (TEF)

Adult to Adult Social Transmission

Adult to Child Social Transmission

Child to Child Social Transmission

Infant Reported behaviours

Assumed intergenerational relationships

Mortality ratios

The Metabolic Equivalent of Task (MET)

Light physical activity

Moderate physical activity

Vigorous physical activity

Screen time

Sleep

Daily intake of Fruit

Daily intake of Vegetables

Daily intake of Grains

Daily intake of Dairy

Daily intake of Meat and Protein

Daily intake of Discretionary foods

Daily intake of Fats and Oils

Daily intake of Sugar Sweetened Beverage

Daily intake of Water

Dailyintake of Over beverages (non-water, non-ssb)

Daily intake of Other foods

Initial fat-mass (%)

Change in Male fat-mass % assumptions

Change in Female fat-mass % assumptions

Proportion of nutrients within food group Inputs

Grains

Vegetables

Fruit

Dairy

Meat and Protein

Fats

Discretionary foods

Sugar-sweetened beverages

Miscellaneous (Other)

Non-sugar-sweetened beverages

Initial BMI Prevalence Inputs

Change in Male Initial BMI Prevalence

Change in Female Initial BMI Prevalence

## R version 4.3.2 (2023-10-31)
## Platform: x86_64-pc-linux-gnu (64-bit)
## Running under: Pop!_OS 20.04 LTS
## 
## Matrix products: default
## BLAS:   /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0 
## LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0
## 
## locale:
##  [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
##  [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
##  [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
##  [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
##  [9] LC_ADDRESS=C               LC_TELEPHONE=C            
## [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       
## 
## time zone: Australia/Sydney
## tzcode source: system (glibc)
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] kableExtra_1.3.4 scales_1.2.1     ggpubr_0.6.0     ggh4x_0.2.5     
##  [5] patchwork_1.1.3  openxlsx_4.2.5.2 lubridate_1.9.2  forcats_1.0.0   
##  [9] stringr_1.5.0    dplyr_1.1.2      purrr_1.0.2      readr_2.1.4     
## [13] tidyr_1.3.0      tibble_3.2.1     ggplot2_3.4.3    tidyverse_2.0.0 
## [17] progress_1.2.2  
## 
## loaded via a namespace (and not attached):
##  [1] gtable_0.3.4      xfun_0.40         bslib_0.5.1       rstatix_0.7.2    
##  [5] tzdb_0.4.0        vctrs_0.6.3       tools_4.3.2       generics_0.1.3   
##  [9] fansi_1.0.4       highr_0.10        pkgconfig_2.0.3   webshot_0.5.5    
## [13] lifecycle_1.0.3   farver_2.1.1      compiler_4.3.2    munsell_0.5.0    
## [17] carData_3.0-5     htmltools_0.5.6   sass_0.4.7        yaml_2.3.7       
## [21] pillar_1.9.0      car_3.1-2         crayon_1.5.2      jquerylib_0.1.4  
## [25] cachem_1.0.8      abind_1.4-5       tidyselect_1.2.0  rvest_1.0.3      
## [29] zip_2.3.0         digest_0.6.33     stringi_1.7.12    bookdown_0.35.1  
## [33] labeling_0.4.2    rmdformats_1.0.4  fastmap_1.1.1     grid_4.3.2       
## [37] colorspace_2.1-0  cli_3.6.1         magrittr_2.0.3    utf8_1.2.3       
## [41] broom_1.0.5       withr_2.5.0       prettyunits_1.1.1 backports_1.4.1  
## [45] timechange_0.2.0  rmarkdown_2.24    httr_1.4.7        ggsignif_0.6.4   
## [49] hms_1.1.3         evaluate_0.21     knitr_1.44        viridisLite_0.4.2
## [53] rlang_1.1.1       Rcpp_1.0.11       glue_1.6.2        xml2_1.3.5       
## [57] svglite_2.1.1     rstudioapi_0.15.0 jsonlite_1.8.7    R6_2.5.1         
## [61] systemfonts_1.0.4